From the Cover: Algorithms,games, and evolution

Even the most seasoned students of evolution, starting with Darwin himself, have occasionally expressed amazement that the mechanism of natural selection has produced the whole of Life as we see it around us. There is a computational way to articulate the same amazement: “What algorithm could possibly achieve all this in a mere three and a half billion years?” In this paper we propose an answer: We demonstrate that in the regime of weak selection, the standard equations of population genetics describing natural selection in the presence of sex become identical to those of a repeated game between genes played according to multiplicative weight updates (MWUA), an algorithm known in computer science to be surprisingly powerful and versatile. MWUA maximizes a tradeoff between cumulative performance and entropy, which suggests a new view on the maintenance of diversity in evolution.Precisely how does selection change the composition of the gene pool from generation to generation? The field of population genetics has developed a comprehensive mathematical framework for answering this and related questions (1). Our analysis in this paper focuses particularly on the regime of weak selection, now a widely used assumption (2, 3). Weak selection assumes that the differences in fitness between genotypes are small relative to the recombination rate, and consequently, through a result due to Nagylaki et al. (4) (see also ref. 1, section II.6.2), evolution proceeds near linkage equilibrium, a regime where the probability of occurrence of a certain genotype involving various alleles is simply the product of the probabilities of each of its alleles. Based on this result, we show that evolution in the regime of weak selection can be formulated as a repeated game, where the recombining loci are the players, the alleles in those loci are the possible actions or strategies available to each player, and the expected payoff at each generation is the expected fitness of an organism across the genotypes that are present in the population. Moreover, and perhaps most importantly, we show that the equations of population genetic dynamics are mathematically equivalent to positing that each locus selects a probability distribution on alleles according to a particular rule which, in the context of the theory of algorithms, game theory, and machine learning, is known as the multiplicative weight updates algorithm (MWUA). MWUA is known in computer science as a simple but surprisingly powerful algorithm (see ref. 5 for a survey). Moreover, there is a dual view of this algorithm: each locus may be seen as selecting its new allele distribution at each generation so as to maximize a certain convex combination of (i) cumulative expected fitness and (ii) the entropy of its distribution on alleles. This connection between evolution, game theory, and algorithms seems to us rife with productive insights; for example, the dual view just mentioned sheds new light on the maintenance of diversity in evolution.Game theory has been applied to evolutionary theory before, to study the evolution of strategic individual behavior (see, e.g., refs. 6, 7). The connection between game theory and evolution that we point out here is at a different level, and arises not in the analysis of strategic individual behavior, but rather in the analysis of the basic population genetic dynamics in the presence of sexual reproduction. The main ingredients of evolutionary game theory, namely strategic individual behavior and conflict between individuals, are extraneous to our analysis.We now state our assumptions and results. We consider an infinite panmictic population of haplotypes involving several unlinked (i.e., fully recombining) loci, where each locus has several alleles. These assumptions are rather standard in the literature. They are made here to simplify exposition and algebra, and there is no a priori reason to believe that they are essential for the results, beyond making them easily accessible. For example, Nagylaki’s theorem (4), which is the main analytical ingredient of our results, holds even in the presence of diploidy and partial recombination.Nagylaki’s theorem states that weak selection in the presence of sex proceeds near the Wright manifold, where the population genetic dynamics becomes (SI Text)xit+1(j)=1Xtxit(j)(Fit(j)),where xit(j) is the frequency of allele j of locus i in the population at generation t, X is a normalizing constant to keep the frequencies summing to 1, and Fit(j) is the mean fitness at time t among genotypes that contain allele j at locus i (see ref. 4 and SI Text). Under the assumption of weak selection, the fitnesses of all genotypes are close to one another, say within the interval [1 − ε, 1 + ε], and so the fitness of genotype g can be written as F_g = 1 + εΔ_g, where ε is the selection strength, assumed here to be small, and Δ_g ∈ [−1, 1] can be called the differential fitness of the genotype. With this in mind, the equation above can be writtenxit+1(j)=1Xtxit(j)(1+ϵΔit(j)),[1]where Δit(j) is the expected differential fitness among genotypes that contain allele j at locus i (see Fig. 1 for an illustration of population genetics at linkage equilibrium).Open in a separate windowFig. 1.Equations of population genetics formulated in the 1930s constitute the standard mathematical way of understanding evolution of a species by tracking the frequencies of various genotypes in a large population. In the simple example shown here, a haploid organism with two genetic loci A and B has three alleles in each of its two loci named A1, A2, A3 and B1, B2, B3 for a total of nine genotypes. In A we show the fitness of each genotype, that is, its expected number of offspring. The fitness numbers shown in A are all close to each other, reflecting weak selection, where the individual alleles’ contributions to fitness are typically minuscule. Initially, each genotype occurs in the population with some frequency; in this particular example these numbers are initially equal (B); naturally, their sum over all nine genotypes is 1 (frequencies are truncated to the fourth decimal digit). C shows how the genotype frequencies evolve in the next generation: each individual of a given genotype produces a number of offspring that is proportional to its fitness shown in A, and the resulting offspring inherits the alleles of its parents with equal probability (because we are assuming, crucially, sexual reproduction). The genotype frequencies in the next generation are shown in C, calculated through the standard recurrence equations of population genetics. We also show in the margins of the table the allele frequencies, obtained by adding the genotype frequencies along the corresponding row or column. Ten generations later, the frequencies are as shown in D.We now introduce the framework of game theory (see Fig. 2 for an illustration) and the MWUA (SI Text), studied in computer science and machine learning, and rediscovered many times over the past half-century; as a result of these multiple rediscoveries, the algorithm is known with various names across subfields: “the experts algorithm” in the theory of algorithms, “Hannan consistency” in economics, “regret minimization” in game theory, “boosting” and “winnow” in artificial intelligence, etc. Here we state it in connection to games, which is only a small part of its applicability (see SI Text for an introduction to the MWUA in connection to the so-called “experts problem” in computer science).Open in a separate windowFig. 2.A simple coordination game is played by two players: the row player, who chooses a row, and the column player, who chooses a column. After the two players make a choice, they both receive (or both pay, in case of a negative entry) the same amount of money, equal to the number at the chosen row and column (A). Coordination games are the simplest possible kind of a game, one in which the strategic interests of all players are completely aligned—that is to say, there is no conflict at all. They are of interest when it is difficult for the players to know these numbers, or to communicate and agree on a mutually beneficial combination (in this example, third row and second column). Notice that this particular coordination game is closely related to the fitness landscape shown in Fig. 1A: If P is a payoff in this game, the corresponding entry of Fig. 1A is equal to 1 + ε ⋅ P, where ε is a small parameter here taken to be 0.01. Suppose that each of the two players chooses each of the three options with some probability, initially 1/3 for all (B); in game theory such probabilistic play is called a mixed action. How do we expect these probabilities to change over repeated play? One famous recipe is the MWUA, in which a player “boosts” the probability of each option by multiplying it by 1 + εG, where G is the expected amount of money this option is going to win the player in the current round of play, and ε is the same small parameter as above. For example, the second action of the row player has G equal to 2 (the average of 3, −1, and 4), and so the probability of playing the second row will be multiplied by 1.02. Then these weights are “renormalized” so they add up to 1, yielding the marginal probabilities shown in C. The probabilities after 10 such rounds of play are shown in D. Comparing now the numbers in the margins of Figs. 1D and ​and2D,2D, we notice that they are essentially the same. This is what we establish mathematically in this paper: the two processes—repeated coordination games played through multiplicative updates, and evolution under weak selection—are essentially identical. This conclusion is of interest because the MWUA is known in computer science to be surprisingly powerful.A game has several players, and each player i has a set A_i of possible actions. Each player also has a utility, capturing the way whereby her actions and the actions of the other players affect this player’s well-being. Formally the utility of a player is a function that maps each combination of actions by the players to a real number (intuitively denoting the player’s gain, in some monetary unit, if all players choose these particular actions). In general, rather than choosing a single action, a player may instead choose a mixed or randomized action, that is, a probabilistic distribution over her action set. Here we only need to consider coordination games, in which all players have the same utility function—that is, the interests of the players are perfectly aligned, and their only challenge is to coordinate their choices effectively. Coordination games are among the simplest games; the only challenge in such a game is for the players to “agree” on a mutually beneficial action.How do the players choose and adjust their choice of randomized (mixed) actions over repeated play? Assume that at time t, player i has mixed action xit, assigning to each action j ∈ A_i the probability xit(j). The MWUA algorithm (5) adjusts the mixed strategy for player i in the next round of the game according to the following rule:xit+1(j)=1Ztxit(j)(1+ϵuit(j)),[2]where Z^t is a normalizing constant designed to ensure that ∑jxit(j)=1, so xit+1 is a probability distribution; ε is a crucial small positive parameter, and uit(j) denotes the expected utility gained by player i choosing action j in the regime of the mixed actions by the other players effective at time t. This algorithm (i) is known to converge to the min–max actions if the game is two-player zero-sum; (ii) is also shown here to converge to equilibrium for the coordination games of interest in the present paper (SI Text, Corollary 5); (iii) is a general “learning algorithm” that has been shown to be very successful in both theory and practice; and (iv) if, instead of games, it is applied to a large variety of optimization problems, including linear programming, convex programming, and network congestion, it provably converges to the optimum quite fast.It can be now checked that the two processes expressed in Eqs. 1 and 2, evolution under natural selection in the presence of sex and multiplicative weight updates in a coordination game, are mathematically identical (SI Text, Theorem 3). That is, the interaction of weak selection and sex is equivalent to the MWUA in a coordination game between loci in which the common utility is the differential fitness of the organism. The parameter ε in the algorithm, which, when small signifies that the algorithm is taking a “longer-term view” of the process to be solved (SI Text), corresponds to the selection strength in evolution, i.e., the magnitude of the differences between the fitness of various genotypes.The MWUA is known in computer science as an extremely simple and yet unexpectedly successful algorithm, which has surprised us time and again by its prowess in solving sophisticated computational problems such as congestion minimization in networks and convex programming in optimization. The observation that multiplicative weight updates in a coordination game are equivalent to evolution under sex and weak selection makes an informative triple connection between three theoretical fields: evolutionary theory, game theory, and the theory of algorithms–machine learning.So far we have presented the MWUA by “how it works” (informally, it boosts alleles proportionally to how well they do in the current mix). There is an alternative way of understanding the MWUA in terms of “what it is optimizing.” That is, we imagine that the allele frequencies of each locus in each generation are the result of a deliberate optimization by the locus of some quantity, and we wish to determine that quantity.Returning to the game formulation, define Uit(j)=∑τ=0tuiτ(j) to be the cumulative utility obtained by player i by playing strategy j over all t first repetitions of the game, and consider the quantity∑jxit(j)Uit(j)−1ϵ∑jxit(j)ln⁡xit(j).[3]The first term is the current (at time t) expected cumulative utility. The second term of 3 is the entropy (expected negative logarithm) of the probability distribution {x_i(j), j = 1, … |A_i|}, multiplied by a large constant 1/?. Suppose now that player i wished to choose the probabilities of actions xit(j)s with the sole goal of maximizing the quantity 3. This is a relatively easy optimization problem, because the quantity 3 to be maximized is strictly concave, and therefore it has a unique maximum, obtained through the Karush–Kuhn–Tucker conditions of optimality (8) (SI Text, section 4):Uit(j)−1ϵ(1+ln⁡xit(j))+μt=0.[Here μ^t is the Lagrange multiplier associated with the constraint ∑jxit(j)=1 seeking to keep the xit(j)s a probability distribution; see SI Text.] Subtracting this equation from its homolog with t replaced by t + 1, and applying the approximation exp(ϵuit(j))≈(1+ϵuit(j)), we obtain the precise Eq. 2 (the normalization Z^t is obtained from μ^t and μ^t+1; see SI Text for the more detailed derivation).Thus, because Eqs. 1 and 2 are identical, we conclude that, in the weak selection regime, natural selection is tantamount to each locus choosing at each generation its allele frequencies in the population so as to maximize the sum of the expected cumulative differential fitness over the alleles, plus the distribution’s entropy. Note that quantity 3 is maximized by genes, not by individuals, and that, interestingly, it is maximized with respect to current frequencies while being dependent (through U^t) on all past frequencies, and although there is some precedent to the use of “historical fitness” (9), its importance in this context is unexpected.This alternative view of selection provides a new insight into an important question in evolutionary biology, namely: How is genetic diversity maintained in the presence of natural selection (10)? That the MWUA process enhances the entropy of the alleles’ distribution (while at the same time optimizes expected cumulative utility) hints at such a mechanism. In fact, entropy is enhanced inversely proportional to s (the quantity corresponding in the population genetics domain to the parameter ε), the selection strength: The weaker the selection, the more it favors high entropy. Naturally, entropy will eventually vanish when the process quiesces at equilibrium: One allele per locus will eventually be fixed, and in fact this equilibrium may be a local, as opposed to global, fitness maximum. However, we believe that it is interesting and significant that the entropy of the allele distribution is favored by selection in the transient; in any event, mutations, environmental changes, and finite population effects are likely to change the process before equilibrium is reached. This new way of understanding the maintenance of variation in evolution (selection as a tradeoff between fitness and entropy maximization) is quite different from previous hypotheses for the maintenance of variation (e.g., refs. 11, 12). Another rather surprising consequence of this characterization is that, under weak selection, all past generations, no matter how distant, have equal influence on the change in the allele mix of the current generation.Our discussion has focused on the evolution of a fixed set of alleles; that is, we have not discussed mutations. Mutations are, of course, paramount in evolution, as they are the source of genetic diversity, and we believe that introducing mutations to the present analysis is an important research direction. Here we focus on the selection process, which is rigorously shown to be tantamount to a tradeoff, for each locus, between maximizing diversity and maximizing expected cumulative fitness.We can now note a simple yet important point. Because multiplicative weight updates by the loci operate in the presence of sex, the triple connection uncovered in this paper is informative for the “queen of problems in evolutionary biology,” namely the role of sex in evolution (13, 14). The notion that the role of sex is the maintenance of diversity has been critiqued (15), because sex does not always increase diversity, and diversity is not always favorable. The MWUA connection sheds new light on the debate, because sex is shown to lead to a tradeoff between increasing entropy and increasing (cumulative) fitness.The connection between the three fields, evolution, game theory, and learning algorithms, described here was not accessible to the founders of the modern synthesis, and we hope that it expands the mathematical tracks that can be traveled in evolution theory.     

Treatment of sarcoid meningitis with radiotherapy

We report a case of sarcoid meningitis associated with grand mal seizures. The patient failed to respond to corticosteroid therapy but showed improvement of symptoms, resulte of computed tomographic scan, and cerebrospinal fluid after receiving low-dose whole-brain irradiation. This form of sarcoldosis is associated with a poor prognosis, and therapy, even with corticosteroids, is generally unsuccessful.     

基于左右制衡理论的镜像疗法在亚急性脑卒中患者的临床应用

目的 验证双侧肢体协调活动在镜像环境下的合理性,以及该治疗范式应用于脑卒中患者的临床可行性。 方法 2018年10月至2019年3月,亚急性期脑卒中患者10例接受数字化镜像疗法双上肢协同任务训练,共4周。治疗前后采用Fugl-Meyer评定量表上肢部分(FMA-UE)、积木障碍盒测试(BBT)、功能独立性测量(FIM)和5次坐立测试(FTSST)进行评定。对参与试验的治疗师和患者进行主观问卷调查。 结果 所有患者均完成试验。治疗后,患者各项观察指标均提高(|Z| > 2.527, P < 0.05)。治疗师和患者均认为该疗法适合脑卒中患者使用。 结论 基于左右制衡理论的镜像疗法对亚急性期脑卒中患者安全、可行,可进一步临床验证疗效。     

医疗集团分级诊疗双向转诊机制建立与实施

建立与完善公立医院与基层医疗卫生服务体系的分工协作,开展双向转诊是新医改工作的重要课题。笔者介绍医疗集团充分发挥医疗联合体的作用,全力推进医院与社区分工协作、双向转诊的实践探索及取得的成效。     

A latent change score approach to understanding dynamic autonomic coordination

Children's self-regulation is a core adaptive system in child development. Physiological indices of regulation, particularly the autonomic nervous system (ANS), have garnered increased attention as an informative level of analysis in regulation research. Cardiography supports the simultaneous examination of both ANS branches via measures of pre-ejection period (PEP) and respiratory sinus arrhythmia (RSA) as indicators of sympathetic and parasympathetic activity, respectively. However, despite their heavily intertwined functions, research examining autonomic coordination across sympathetic and parasympathetic systems is scarce. Moreover, extant efforts have favored static, mean level reactivity analyses, despite the dynamic nature of ANS regulation and the availability of analytic tools that can model these processes across time. This study drew on a sample of 198 six-year-old children from a diverse community sample (49.5% female, 43.9% Latinx) to examine dynamic autonomic coordination using bivariate latent change score modeling to evaluate bidirectional influences of sympathetic and parasympathetic activity over the course of a challenging puzzle completion task. Results indicated that children evidenced reciprocal sympathetic activation (i.e., PEP attenuation and RSA withdrawal) across the challenge task, and these regulatory responses were characterized by a temporally leading influence of PEP on lagging changes in RSA. The current findings contribute to our understanding of children's autonomic coordination while illustrating a novel analytic technique to advance ongoing efforts to understand the etiology and developmental significance of children's physiological self-regulation.     

Real-time biofeedback integrated into neuromuscular training reduces high-risk knee biomechanics and increases functional brain connectivity: A preliminary longitudinal investigation

Prospective evidence indicates that functional biomechanics and brain connectivity may predispose an athlete to an anterior cruciate ligament injury, revealing novel neural linkages for targeted neuromuscular training interventions. The purpose of this study was to determine the efficacy of a real-time biofeedback system for altering knee biomechanics and brain functional connectivity. Seventeen healthy, young, physically active female athletes completed 6 weeks of augmented neuromuscular training (aNMT) utilizing real-time, interactive visual biofeedback and 13 served as untrained controls. A drop vertical jump and resting state functional magnetic resonance imaging were separately completed at pre- and posttest time points to assess sensorimotor adaptation. The aNMT group had a significant reduction in peak knee abduction moment (pKAM) compared to controls (p = .03, d = 0.71). The aNMT group also exhibited a significant increase in functional connectivity between the right supplementary motor area and the left thalamus (p = .0473 after false discovery rate correction). Greater percent change in pKAM was also related to increased connectivity between the right cerebellum and right thalamus for the aNMT group (p = .0292 after false discovery rate correction, r² = .62). No significant changes were observed for the controls (ps > .05). Our data provide preliminary evidence of potential neural mechanisms for aNMT-induced motor adaptations that reduce injury risk. Future research is warranted to understand the role of neuromuscular training alone and how each component of aNMT influences biomechanics and functional connectivity.     

The effects of eye coordination during deep cervical flexor training on the thickness of the cervical flexors

[Purpose] The purpose of this study was to identify changes in the thicknesses of the cervical flexors according to eye coordination during deep cervical flexor training. [Subjects and Methods] Twenty normal adults were randomly selected, and during their deep cervical flexor training and eye tracking, the thicknesses of the longus colli and the sternocleidomastoid were measured using ultrasonic waves. [Results] The thickness of the longus colli statistically significantly increased when deep cervical flexor training and eye coordination were performed simultaneously. However, the thickness of the sternocleidomastoid did not show statistically significant differences according to eye coordination. [Conclusion] Eye coordination during deep cervical flexor training is likely to increase the thickness of the longus colli selectively.     

Cerebellar premotor output neurons collateralize to innervate the cerebellar cortex

Motor commands computed by the cerebellum are hypothesized to use corollary discharge, or copies of outgoing commands, to accelerate motor corrections. Identifying sources of corollary discharge, therefore, is critical for testing this hypothesis. Here we verified that the pathway from the cerebellar nuclei to the cerebellar cortex in mice includes collaterals of cerebellar premotor output neurons, mapped this collateral pathway, and identified its postsynaptic targets. Following bidirectional tracer injections into a distal target of the cerebellar nuclei, the ventrolateral thalamus, we observed retrogradely labeled somata in the cerebellar nuclei and mossy fiber terminals in the cerebellar granule layer, consistent with collateral branching. Corroborating these observations, bidirectional tracer injections into the cerebellar cortex retrogradely labeled somata in the cerebellar nuclei and boutons in the ventrolateral thalamus. To test whether nuclear output neurons projecting to the red nucleus also collateralize to the cerebellar cortex, we used a Cre‐dependent viral approach, avoiding potential confounds of direct red nucleus‐to‐cerebellum projections. Injections of a Cre‐dependent GFP‐expressing virus into Ntsr1‐Cre mice, which express Cre selectively in the cerebellar nuclei, retrogradely labeled somata in the interposed nucleus, and putative collateral branches terminating as mossy fibers in the cerebellar cortex. Postsynaptic targets of all labeled mossy fiber terminals were identified using immunohistochemical Golgi cell markers and electron microscopic profiles of granule cells, indicating that the collaterals of nuclear output neurons contact both Golgi and granule cells. These results clarify the organization of a subset of nucleocortical projections that constitute an experimentally accessible corollary discharge pathway within the cerebellum. J. Comp. Neurol. 523:2254–2271, 2015. © 2015 Wiley Periodicals, Inc.     

Fundamental movement skills proficiency in children with developmental coordination disorder: does physical self-concept matter?

Purpose: The purpose of this study was to (1) examine differences in fundamental movement skills (FMS) proficiency, physical self-concept, and physical activity in children with and without developmental coordination disorder (DCD), and (2) determine the association of FMS proficiency with physical self-concept while considering key confounding factors. Method: Participants included 43 children with DCD and 87 age-matched typically developing (TD) children. FMS proficiency was assessed using the Test of Gross Motor Development – second edition. Physical self-concept and physical activity were assessed using self-report questionnaires. A two-way (group by gender) ANCOVA was used to determine whether between-group differences existed in FMS proficiency, physical self-concept, and physical activity after controlling for age and BMI. Partial correlations and hierarchical multiple regression models were used to examine the relationship between FMS proficiency and physical self-concept. Results: Compared with their TD peers, children with DCD displayed less proficiency in various components of FMS and viewed themselves as being less competent in physical coordination, sporting ability, and physical health. Physical coordination was a significant predictor of ability in object control skills. DCD status and gender were significant predictors of FMS proficiency. Conclusions: Future FMS interventions should target children with DCD and girls, and should emphasize improving object control skills proficiency and physical coordination.

• Implications for Rehabilitation

• Children with DCD tend to have not only lower FMS proficiency than age-matched typically developing children but also lower physical self-concept.

• Self-perceptions of physical coordination by children with DCD are likely to be valuable contributors to development of object control skills. This may then help to develop their confidence in performing motor skills.

• Children with DCD need supportive programs that facilitate the development of object control skills. Efficacy of training programs may be improved if children experience a greater sense of control and success when performing object control skills.